Gaze-aware tone mapping and chromatic adaptation

ABSTRACT

According to examples, a system may use gaze direction information to perform tone mapping or chromatic adaptation for virtual reality (VR) environments. The system may include a processor and a memory storing instructions. When the processor executes the instructions, the system may determine a user focus location within a physical environment. An image corresponding to the physical environment may be received. The system may determine a visual property of the physical environment corresponding to the user focus location. The system may modify a color mapping of a region of the image based on the determined visual property to generate an output image.

TECHNICAL FIELD

This patent application relates generally to display image renderingtechniques, and more specifically, to systems and methods usinggaze-aware tone mapping and chromatic adaptation.

BACKGROUND

With recent advances in technology, prevalence and proliferation ofcontent creation and delivery has increased greatly in recent years. Inparticular, interactive content such as virtual reality (VR) content,augmented reality (AR) content, mixed reality (MR) content, and contentwithin and associated with a real and/or virtual environment (e.g., a“metaverse”) has become appealing to consumers.

To facilitate delivery of this and other related content, serviceproviders have endeavored to provide various forms of wearable displaysystems. One such example may be a head-mounted display (HMD) device,such as a wearable eyewear, a wearable headset, or eyeglasses. In someexamples, the head-mounted display (HMD) device may project or directlight to form a first image and a second image, and with these images,to generate “binocular” vision for viewing by a user. A head-mounteddisplay (HMD) device may offer a wider field of view (FOV) than otherdisplays. Due to optical design and hardware limitations, however,achieving uniform color across the field of view can be difficult. Forexample, some display hardware may exhibit color changes when viewedfrom different viewing angles. Optical layers and/or lenses between thedisplay hardware and the user's eye may also exhibit color changes whenviewed from different viewing angles.

BRIEF DESCRIPTION OF DRAWINGS

Features of the present disclosure are illustrated by way of example andnot limited in the following figures, in which like numerals indicatelike elements. One skilled in the art will readily recognize from thefollowing that alternative examples of the structures and methodsillustrated in the figures can be employed without departing from theprinciples described herein.

FIG. 1 illustrates a block diagram of an artificial reality systemenvironment including a near-eye display, according to an example.

FIG. 2 illustrates a perspective view of a near-eye display in the formof a head-mounted display (HMD) device, according to an example.

FIG. 3 illustrates a perspective view of a near-eye display in the formof a pair of glasses, according to an example.

FIG. 4 illustrates a block diagram of a color correction system,according to an example.

FIG. 5 is a diagram illustrating tone mapping including brightnessadjustment, according to an example.

FIG. 6 is a diagram illustrating chromatic adaptation including localwhite balancing.

FIG. 7 illustrates a diagram of an example of veiling glare.

FIG. 8 illustrates a diagram showing examples of dynamic colorcorrection (DCC) and dynamic brightness correction (DBC), according tosome examples.

FIG. 9 is a flow diagram illustrating an example method for usinggaze-aware tone mapping and chromatic adaptation, according to someexamples.

DETAILED DESCRIPTION

For simplicity and illustrative purposes, the present application isdescribed by referring mainly to examples thereof. In the followingdescription, numerous specific details are set forth in order to providea thorough understanding of the present application. It will be readilyapparent, however, that the present application may be practiced withoutlimitation to these specific details. In other instances, some methodsand structures readily understood by one of ordinary skill in the arthave not been described in detail so as not to unnecessarily obscure thepresent application. As used herein, the terms “a” and “an” are intendedto denote at least one of a particular element, the term “includes”means includes but not limited to, the term “including” means includingbut not limited to, and the term “based on” means based at least in parton.

Virtual reality (VR) and augmented reality (AR) displays may offer asignificantly wider field of view (FOV) than traditional displays,creating an immersive experience for a user. However, due to opticaldesign and hardware limitations, it may be difficult to achieve uniformcolor across the field of view. For example, display hardware, such asorganic light emitting diodes (OLEDs), may exhibit changes in perceivedcolor when viewed from different viewing angles. Optical layers and/orlenses between the display and the human eye may also contribute tocolor distortion. As another example, viewing optics in virtual reality(VR) systems (e.g., pancake optics) may exhibit ghosting and veilingglare issues due to complex optical layers and coatings.

In some examples of the present disclosure, information relating to gazedirection may be used, e.g., with other information, to perform colorcorrection in a virtual reality (VR), augmented reality (AR), or mixedreality (MR) environment. For example, a region in a field of view of adisplayed virtual reality (VR) environment may be characterized bybright content. In some examples, the brightness of the region may bemodified, e.g., adjusted downward, based on a user focus location towardwhich the user's gaze is directed. The brightness of the region may bemodified based on the optical design of the head-mounted display (HMD)device. As another example, the user focus location may be used as areference for chromatic adaptation of the region, e.g., adjusting awhite point of the region so that the color appearance of the region isrepresentative of the color appearance of a corresponding physicalenvironment.

According to examples, a system may use gaze direction information toperform tone mapping or chromatic adaptation for virtual reality (VR)environments. The system may include a processor and a memory storinginstructions. When the processor executes the instructions, the systemmay determine a user focus location within a physical environment. Animage corresponding to the physical environment may be received. Thesystem may determine a visual property of the physical environmentcorresponding to the user focus location. The system may modify a colormapping of a region of the image based on the determined visual propertyto generate an output image.

Brightness and color uniformity are design considerations in opticaldesign. During optical design processes, certain design considerationsare balanced, including, for example, brightness uniformity, coloruniformity, sharpness, field of view (FOV), resolution, distortionstability, virtual image distance, and the like. Using informationregarding gaze direction to adjust brightness and/or color uniformitydynamically may allow some design considerations to be relaxed.Performance may be improved in some aspects, such as higher resolutionor a wider field of view (FOV).

FIG. 1 illustrates a block diagram of an artificial reality systemenvironment 100 including a near-eye display, according to an example.As used herein, a “near-eye display” may refer to a device (e.g., anoptical device) that may be in close proximity to a user's eye. As usedherein, “artificial reality” may refer to aspects of, among otherthings, a “metaverse” or an environment of real and virtual elements,and may include use of technologies associated with virtual reality(VR), augmented reality (AR), and/or mixed reality (MR). As used hereina “user” may refer to a user or wearer of a “near-eye display.”

As shown in FIG. 1 , the artificial reality system environment 100 mayinclude a near-eye display 120, an optional external imaging device 150,and an optional input/output interface 140, each of which may be coupledto a console 110. The console 110 may be optional in some instances asthe functions of the console 110 may be integrated into the near-eyedisplay 120. In some examples, the near-eye display 120 may be ahead-mounted display (HMD) that presents content to a user.

In some instances, for a near-eye display system, it may generally bedesirable to expand an eye box, reduce display haze, improve imagequality (e.g., resolution and contrast), reduce physical size, increasepower efficiency, and increase or expand field of view (FOV). As usedherein, “field of view” (FOV) may refer to an angular range of an imageas seen by a user, which is typically measured in degrees as observed byone eye (for a monocular HMD) or both eyes (for binocular HMDs). Also,as used herein, an “eye box” may be a two-dimensional box that may bepositioned in front of the user's eye from which a displayed image froman image source may be viewed.

In some examples, in a near-eye display system, light from a surroundingenvironment may traverse a “see-through” region of a waveguide display(e.g., a transparent substrate) to reach a user's eyes. For example, ina near-eye display system, light of projected images may be coupled intoa transparent substrate of a waveguide, propagate within the waveguide,and be coupled or directed out of the waveguide at one or more locationsto replicate exit pupils and expand the eye box.

In some examples, the near-eye display 120 may include one or more rigidbodies, which may be rigidly or non-rigidly coupled to each other. Insome examples, a rigid coupling between rigid bodies may cause thecoupled rigid bodies to act as a single rigid entity, while in otherexamples, a non-rigid coupling between rigid bodies may allow the rigidbodies to move relative to each other.

In some examples, the near-eye display 120 may be implemented in anysuitable form-factor, including an HMD, a pair of glasses, or othersimilar wearable eyewear or device. Examples of the near-eye display 120are further described below with respect to FIGS. 2 and 3 .Additionally, in some examples, the functionality described herein maybe used in an HMD or headset that may combine images of an environmentexternal to the near-eye display 120 and artificial reality content(e.g., computer-generated images). Therefore, in some examples, thenear-eye display 120 may augment images of a physical, real-worldenvironment external to the near-eye display 120 with generated and/oroverlaid digital content (e.g., images, video, sound, etc.) to presentan augmented reality to a user.

In some examples, the near-eye display 120 may include any number ofdisplay electronics 122, display optics 124, and an eye-tracking unit130. In some examples, the near eye display 120 may also include one ormore locators 126, one or more position sensors 128, and an inertialmeasurement unit (IMU) 132. In some examples, the near-eye display 120may omit any of the eye-tracking unit 130, the one or more locators 126,the one or more position sensors 128, and the inertial measurement unit(IMU) 132, or may include additional elements.

In some examples, the display electronics 122 may display or facilitatethe display of images to the user according to data received from, forexample, the optional console 110. In some examples, the displayelectronics 122 may include one or more display panels. In someexamples, the display electronics 122 may include any number of pixelsto emit light of a predominant color such as red, green, blue, white, oryellow. In some examples, the display electronics 122 may display athree-dimensional (3D) image, e.g., using stereoscopic effects producedby two-dimensional panels, to create a subjective perception of imagedepth.

In some examples, the display optics 124 may display image contentoptically (e.g., using optical waveguides and/or couplers) or magnifyimage light received from the display electronics 122, correct opticalerrors associated with the image light, and/or present the correctedimage light to a user of the near-eye display 120. In some examples, thedisplay optics 124 may include a single optical element or any number ofcombinations of various optical elements as well as mechanical couplingsto maintain relative spacing and orientation of the optical elements inthe combination. In some examples, one or more optical elements in thedisplay optics 124 may have an optical coating, such as ananti-reflective coating, a reflective coating, a filtering coating,and/or a combination of different optical coatings.

In some examples, the display optics 124 may also be designed to correctone or more types of optical errors, such as two-dimensional opticalerrors, three-dimensional optical errors, or any combination thereof.Examples of two-dimensional errors may include barrel distortion,pincushion distortion, longitudinal chromatic aberration, and/ortransverse chromatic aberration. Examples of three-dimensional errorsmay include spherical aberration, chromatic aberration field curvature,and astigmatism.

In some examples, the one or more locators 126 may be objects located inspecific positions relative to one another and relative to a referencepoint on the near-eye display 120. In some examples, the optionalconsole 110 may identify the one or more locators 126 in images capturedby the optional external imaging device 150 to determine the artificialreality headset's position, orientation, or both. The one or morelocators 126 may each be a light-emitting diode (LED), a corner cubereflector, a reflective marker, a type of light source that contrastswith an environment in which the near-eye display 120 operates, or anycombination thereof.

In some examples, the external imaging device 150 may include one ormore cameras, one or more video cameras, any other device capable ofcapturing images including the one or more locators 126, or anycombination thereof. The optional external imaging device 150 may beconfigured to detect light emitted or reflected from the one or morelocators 126 in a field of view of the optional external imaging device150.

In some examples, the one or more position sensors 128 may generate oneor more measurement signals in response to motion of the near-eyedisplay 120. Examples of the one or more position sensors 128 mayinclude any number of accelerometers, gyroscopes, magnetometers, and/orother motion-detecting or error-correcting sensors, or any combinationthereof.

In some examples, the inertial measurement unit (IMU) 132 may be anelectronic device that generates fast calibration data based onmeasurement signals received from the one or more position sensors 128.The one or more position sensors 128 may be located external to theinertial measurement unit (IMU) 132, internal to the inertialmeasurement unit (IMU) 132, or any combination thereof. Based on the oneor more measurement signals from the one or more position sensors 128,the inertial measurement unit (IMU) 132 may generate fast calibrationdata indicating an estimated position of the near-eye display 120 thatmay be relative to an initial position of the near-eye display 120. Forexample, the inertial measurement unit (IMU) 132 may integratemeasurement signals received from accelerometers over time to estimate avelocity vector and integrate the velocity vector over time to determinean estimated position of a reference point on the near-eye display 120.Alternatively, the inertial measurement unit (IMU) 132 may provide thesampled measurement signals to the optional console 110, which maydetermine the fast calibration data.

The eye-tracking unit 130 may include one or more eye-tracking systems.As used herein, “eye tracking” may refer to determining an eye'sposition or relative position, including orientation, location, and/orgaze of a user's eye. In some examples, an eye-tracking system mayinclude an imaging system that captures one or more images of an eye andmay optionally include a light emitter, which may generate light that isdirected to an eye such that light reflected by the eye may be capturedby the imaging system. In other examples, the eye-tracking unit 130 maycapture reflected radio waves emitted by a miniature radar unit. Thesedata associated with the eye may be used to determine or predict eyeposition, orientation, movement, location, and/or gaze.

In some examples, the near-eye display 120 may use the orientation ofthe eye to introduce depth cues (e.g., blur image outside of the user'smain line of sight), collect heuristics on the user interaction in thevirtual reality (VR) media (e.g., time spent on any particular subject,object, or frame as a function of exposed stimuli), some other functionsthat are based in part on the orientation of at least one of the user'seyes, or any combination thereof. In some examples, because theorientation may be determined for both eyes of the user, theeye-tracking unit 130 may be able to determine where the user is lookingor predict any user patterns, etc.

In some examples, the input/output interface 140 may be a device thatallows a user to send action requests to the optional console 110. Asused herein, an “action request” may be a request to perform aparticular action. For example, an action request may be to start or toend an application or to perform a particular action within theapplication. The input/output interface 140 may include one or moreinput devices. Example input devices may include a keyboard, a mouse, agame controller, a glove, a button, a touch screen, or any othersuitable device for receiving action requests and communicating thereceived action requests to the optional console 110. In some examples,an action request received by the input/output interface 140 may becommunicated to the optional console 110, which may perform an actioncorresponding to the requested action.

In some examples, the optional console 110 may provide content to thenear-eye display 120 for presentation to the user in accordance withinformation received from one or more of external imaging device 150,the near-eye display 120, and the input/output interface 140. Forexample, in the example shown in FIG. 1 , the optional console 110 mayinclude an application store 112, a headset tracking module 114, avirtual reality engine 116, and an eye-tracking module 118. Someexamples of the optional console 110 may include different or additionalmodules than those described in conjunction with FIG. 1 . Functionsfurther described below may be distributed among components of theoptional console 110 in a different manner than is described here.

In some examples, the optional console 110 may include a processor and anon-transitory computer-readable storage medium storing instructionsexecutable by the processor. The processor may include multipleprocessing units executing instructions in parallel. The non-transitorycomputer-readable storage medium may be any memory, such as a hard diskdrive, a removable memory, or a solid-state drive (e.g., flash memory ordynamic random access memory (DRAM)). In some examples, the modules ofthe optional console 110 described in conjunction with FIG. 1 may beencoded as instructions in the non-transitory computer-readable storagemedium that, when executed by the processor, cause the processor toperform the functions further described below. It should be appreciatedthat the optional console 110 may or may not be needed or the optionalconsole 110 may be integrated with or separate from the near-eye display120.

In some examples, the application store 112 may store one or moreapplications for execution by the optional console 110. An applicationmay include a group of instructions that, when executed by a processor,generates content for presentation to the user. Examples of theapplications may include gaming applications, conferencing applications,video playback application, or other suitable applications.

In some examples, the headset tracking module 114 may track movements ofthe near-eye display 120 using slow calibration information from theexternal imaging device 150. For example, the headset tracking module114 may determine positions of a reference point of the near-eye display120 using observed locators from the slow calibration information and amodel of the near-eye display 120. Additionally, in some examples, theheadset tracking module 114 may use portions of the fast calibrationinformation, the slow calibration information, or any combinationthereof, to predict a future location of the near-eye display 120. Insome examples, the headset tracking module 114 may provide the estimatedor predicted future position of the near-eye display 120 to the virtualreality engine 116.

In some examples, the virtual reality engine 116 may executeapplications within the artificial reality system environment 100 andreceive position information of the near-eye display 120, accelerationinformation of the near-eye display 120, velocity information of thenear-eye display 120, predicted future positions of the near-eye display120, or any combination thereof from the headset tracking module 114. Insome examples, the virtual reality engine 116 may also receive estimatedeye position and orientation information from the eye-tracking module118. Based on the received information, the virtual reality engine 116may determine content to provide to the near-eye display 120 forpresentation to the user.

In some examples, the eye-tracking module 118 may receive eye-trackingdata from the eye-tracking unit 130 and determine the position of theuser's eye based on the eye tracking data. In some examples, theposition of the eye may include an eye's orientation, location, or bothrelative to the near-eye display 120 or any element thereof. So, inthese examples, because the eye's axes of rotation change as a functionof the eye's location in its socket, determining the eye's location inits socket may allow the eye-tracking module 118 to more accuratelydetermine the eye's orientation.

In some examples, a location of a projector of a display system may beadjusted to enable any number of design modifications. For example, insome instances, a projector may be located in front of a viewer's eye(i.e., “front-mounted” placement). In a front-mounted placement, in someexamples, a projector of a display system may be located away from auser's eyes (i.e., “world-side”). In some examples, a head-mounteddisplay (HMD) device may utilize a front-mounted placement to propagatelight towards a user's eye(s) to project an image.

FIG. 2 illustrates a perspective view of a near-eye display in the formof a head-mounted display (HMD) device 200, according to an example. Insome examples, the HMD device 200 may be a part of a virtual reality(VR) system, an augmented reality (AR) system, a mixed reality (MR)system, another system that uses displays or wearables, or anycombination thereof. In some examples, the HMD device 200 may include abody 220 and a head strap 230. FIG. 2 shows a bottom side 223, a frontside 225, and a left side 227 of the body 220 in the perspective view.In some examples, the head strap 230 may have an adjustable orextendible length. In particular, in some examples, there may be asufficient space between the body 220 and the head strap 230 of the HMDdevice 200 for allowing a user to mount the HMD device 200 onto theuser's head. For example, the length of the head strap 230 may beadjustable to accommodate a range of user head sizes. In some examples,the HMD device 200 may include additional, fewer, and/or differentcomponents.

In some examples, the HMD device 200 may present, to a user, media orother digital content including virtual and/or augmented views of aphysical, real-world environment with computer-generated elements.Examples of the media or digital content presented by the HMD device 200may include images (e.g., two-dimensional (2D) or three-dimensional (3D)images), videos (e.g., 2D or 3D videos), audio, or any combinationthereof. In some examples, the images and videos may be presented toeach eye of a user by one or more display assemblies (not shown in FIG.2 ) enclosed in the body 220 of the HMD device 200.

In some examples, the HMD device 200 may include various sensors (notshown), such as depth sensors, motion sensors, position sensors, and/oreye tracking sensors. Some of these sensors may use any number ofstructured or unstructured light patterns for sensing purposes. In someexamples, the HMD device 200 may include an input/output interface 140for communicating with a console 110, as described with respect to FIG.1 . In some examples, the HMD device 200 may include a virtual realityengine (not shown), but similar to the virtual reality engine 116described with respect to FIG. 1 , that may execute applications withinthe HMD device 200 and receive depth information, position information,acceleration information, velocity information, predicted futurepositions, or any combination thereof of the HMD device 200 from thevarious sensors.

In some examples, the information received by the virtual reality engine116 may be used for producing a signal (e.g., display instructions) tothe one or more display assemblies. In some examples, the HMD device 200may include locators (not shown), but similar to the virtual locators126 described in FIG. 1 , which may be located in fixed positions on thebody 220 of the HMD device 200 relative to one another and relative to areference point. Each of the locators may emit light that is detectableby an external imaging device. This may be useful for the purposes ofhead tracking or other movement/orientation. It should be appreciatedthat other elements or components may also be used in addition or inlieu of such locators.

It should be appreciated that in some examples, a projector mounted in adisplay system may be placed near and/or closer to a user's eye (i.e.,“eye-side”). In some examples, and as discussed herein, a projector fora display system shaped liked eyeglasses may be mounted or positioned ina temple arm (i.e., a top far corner of a lens side) of the eyeglasses.It should be appreciated that, in some instances, utilizing aback-mounted projector placement may help to reduce size or bulkiness ofany required housing required for a display system, which may alsoresult in a significant improvement in user experience for a user.

FIG. 3 is a perspective view of a near-eye display 300 in the form of apair of glasses (or other similar eyewear), according to an example. Insome examples, the near-eye display 300 may be a specific example ofnear-eye display 120 of FIG. 1 , and may be configured to operate as avirtual reality display, an augmented reality (AR) display, and/or amixed reality (MR) display.

In some examples, the near-eye display 300 may include a frame 305 and adisplay 310. In some examples, the display 310 may be configured topresent media or other content to a user. In some examples, the display310 may include display electronics and/or display optics, similar tocomponents described with respect to FIGS. 1-2 . For example, asdescribed above with respect to the near-eye display 120 of FIG. 1 , thedisplay 310 may include a liquid crystal display (LCD) display panel, alight-emitting diode (LED) display panel, or an optical display panel(e.g., a waveguide display assembly). In some examples, the display 310may also include any number of optical components, such as waveguides,gratings, lenses, mirrors, etc.

In some examples, the near-eye display 300 may further include varioussensors 350 a, 350 b, 350 c, 350 d, and 350 e on or within a frame 305.In some examples, the various sensors 350 a-350 e may include any numberof depth sensors, motion sensors, position sensors, inertial sensors,and/or ambient light sensors, as shown. In some examples, the varioussensors 350 a-350 e may include any number of image sensors configuredto generate image data representing different fields of views in one ormore different directions. In some examples, the various sensors 350a-350 e may be used as input devices to control or influence thedisplayed content of the near-eye display 300, and/or to provide aninteractive virtual reality (VR), augmented reality (AR), and/or mixedreality (MR) experience to a user of the near-eye display 300. In someexamples, the various sensors 350 a-350 e may also be used forstereoscopic imaging or other similar application.

In some examples, the near-eye display 300 may further include one ormore illuminators 330 to project light into a physical environment. Theprojected light may be associated with different frequency bands (e.g.,visible light, infra-red light, ultra-violet light, etc.), and may servevarious purposes. In some examples, the one or more illuminator(s) 330may be used as locators, such as the one or more locators 126 describedabove with respect to FIGS. 1-2 .

In some examples, the near-eye display 300 may also include a camera 340or other image capture unit. The camera 340, for instance, may captureimages of the physical environment in the field of view. In someinstances, the captured images may be processed, for example, by avirtual reality engine (e.g., the virtual reality engine 116 of FIG. 1 )to add virtual objects to the captured images or modify physical objectsin the captured images, and the processed images may be displayed to theuser by the display 310 for augmented reality (AR) and/or mixed reality(MR) applications.

FIG. 4 illustrates a block diagram of a color correction system 400configured to use gaze direction information to perform tone mapping orchromatic adaptation for virtual reality (VR) environments, inaccordance with various examples. The color correction system 400 mayinclude one or more computing platforms 402. The one or more computingplatforms 402 may be communicatively coupled with one or more remoteplatforms 404. In some examples, users may access the color correctionsystem 400 via the remote platforms 404.

In some examples, the one or more computing platforms 402 may beconfigured by computer-readable instructions 406. Computer-readableinstructions 406 may include modules. The modules may be implemented asone or more of functional logic, hardware logic, electronic softwaremodules, and the like. The modules may include one or more of a dataobtaining module 408, a gaze analysis module 410, a color analysismodule 412, and a pixel modifier module 414.

In some examples, the data obtaining module 408 may receive an imagecorresponding to a physical environment, e.g., in which the colorcorrection system 400 is located. For example, the color correctionsystem 400 may be embodied in a head-mounted display (HMD) device. Thehead-mounted display (HMD) device may include one or more camera(s) orimage sensor(s) that capture images representing the physicalenvironment. The images may be still images. The images may be framesextracted from a video feed. The head-mounted display (HMD) device mayinclude a display (e.g., a screen) that displays the images or modifiedimages.

In some examples, the gaze analysis module 410 may determine a userfocus location within the physical environment on which the user'sattention is focused. The color correction system 400 may use gazedirection as a proxy for the user's attention, e.g., the colorcorrection system 400 may determine that the user's attention is focusedon a location at which the user's gaze is directed. In some examples,the color correction system 400 may determine the user focus locationbased at least in part on other proxies for the user's attention, suchas a location of a hand gesture or a location of a pointing device. Asanother example, the color correction system 400 may determine the userfocus location based at least in part on the location of an object thatis present in the physical environment. For example, if a prominentobject is in the physical environment, the color correction system 400may determine that tone mapping should be centered on that object, evenif the user may not be looking at the object. In some examples, the gazeanalysis module 410 determines a gaze direction of the user. Forexample, the gaze analysis module 410 may receive image data fromuser-facing cameras or image sensors. The image data may represent animage or images that include pixels representing the user's pupils.Based on the position of the user's pupils, the gaze analysis module 410may determine the direction of the user's gaze. In some examples, theuser's eyes may be illuminated, for example, by LASER or near infrared(nIR) illumination sources, such as light emitting diodes (LEDs). Theimage represented by the image data may include pixels representingreflections from the user's eyes (e.g., glints) that may be used todetermine the direction of the user's gaze.

In some examples, the color analysis module 412 determines a visualproperty of the physical environment corresponding to the user focuslocation. For example, the color analysis module 412 may receive imagedata representing the physical environment from a world-facing camera orimage sensor. The color analysis module 412 may analyze color valuesassociated with one or more pixels in the image data to determine acolor value associated with the user focus location in the physicalenvironment to which the user's gaze is directed. The brightness of theuser focus location may be determined.

In some examples, the color analysis module 412 may select a size of theuserfocus location. For example, the color analysis module 412 mayselect a size of the area that is to serve as a basis for tone mappingand/or chromatic adaptation. In some examples, the color analysis module412 may apply an averaging or filtering function to the image datarepresenting the physical environment. For example, the color analysismodule 412 may apply a filtering function to account for noise and/ornatural micro saccade from the eye tracker. In some examples, the coloranalysis module 412 may differentiate between different eye movementssuch as saccade, micro saccade, slow pursuit, and/or vestibulo ocularreflex (VOR). The color analysis module 412 may apply differentfiltering functions to account for each type of eye movement. Forexample, the color analysis module 412 may apply little or no averagingto the image data in the case of saccade so that changes may be appliedmore quickly.

In some examples, the pixel modifier module 414 modifies a color mappingof a region of the image corresponding to the physical environment basedon the determined visual property to generate an output image. Forexample, the pixel modifier module 414 may use the color values from thearea around the user focus location to perform color correction on aregion of the image.

In some examples, modifying the color mapping may include modifying atone mapping of the region of the image. For example, the pixel modifiermodule 414 may perform dynamic brightness adjustment. FIG. 5 is adiagram illustrating tone mapping including brightness adjustment,according to an example. As illustrated in FIG. 5 , an image 502represents a real world scene including a wide range of brightnessvalues. An image 504 represents an example virtual reality (VR)representation of the real world scene illustrated in image 502. It willbe appreciated that, in image 504, the dynamic range of the image isreduced relative to the image 502. For example, the bright area on theright side of image 504 appears significantly dimmer than thecorresponding area in image 502. Some displays for high dynamic range(HDR) images are capable of high absolute brightness (e.g., nits). Insome examples, dynamic brightness adjustment may be performed withoutthe need for a display with high absolute brightness capabilities.

For example, as represented in an image 506, the gaze analysis module410 may determine that the user's gaze is directed to a user focuslocation 508. The pixel modifier module 414 may modify the tone mappingof image 506 based on an area around the user focus location 508.Because the area around the user focus location 508 is relatively dark,the pixel modifier module 414 may modify the tone mapping of image 506to increase the brightness of image 506.

As another example, as represented in an image 510, the gaze analysismodule 410 may determine that the user's gaze is directed to a userfocus location 512. The pixel modifier module 414 may modify the tonemapping of image 510 based on an area around the user focus location512. Because the area around the user focus location 512 is relativelybright, the pixel modifier module 414 may modify the tone mapping ofimage 510 to decrease the brightness of image 510.

In some examples, due to optical design and hardware limitations, it maybe difficult to achieve uniform color across the field of view (FOV).Display hardware (e.g., organic light emitting diodes (OLEDs)) mayexhibit color changes when viewed from different viewing angles. In someexamples, brightness correction may be implemented as part of colorcorrection. Brightness correction may be achieved by emission duty orpersistence adjustment and/or backlight spatial adjustment. For example,within each frame (e.g., 90 frames per second (fps) for virtualreality), the display or backlight may not be constantly on. The displayor backlight may have a duty cycle of, for example, 10% (e.g., 1 ms),20%, or 30%. In some examples, backlight spatial adjustment may beachieved by spatially varying the brightness across the surface of thedisplay. This may be achieved in a backlight with arrayed LEDs.Brightness correction may be spatially slow, e.g., there may be few orno sharp corrections. In some examples, brightness correction may beperformed every 10° across the field of view (FOV) with a smoothtransition, such as a B spline surface, e.g., the surface 810 of FIG. 8.

In some examples, modifying the color mapping may include performingchromatic adaptation of the region of the image. For example, the pixelmodifier module 414 may perform local white balancing (e.g., adjust awhite point) based on an area around the user focus location. FIG. 6 isa diagram illustrating chromatic adaptation including local whitebalancing, according to an example.

As illustrated in FIG. 6 , an image 602 represents a real world sceneincluding two light sources 604, 606. The light source 604 may have acolor temperature of, e.g., 2700 K. The light source 606 may have acolor temperature of, e.g., 6500 K. Image 602 may exhibit global whitebalancing. In some examples, white balancing may be performed by usingthe color temperature of an area of maximum brightness in the scene as areference. This may lead to unsatisfactory results. Images 608, 610 mayrepresent the scene illustrated in image 602 with local white balancing.For example, the white point of the area near the user focus locationmay be adjusted according to the color value of a corresponding locationin the physical environment and/or adaptation of the human eye. Forexample, if the user gazes at the light source 604, local whitebalancing may be performed using the color value of the light source 604as the white point, as shown in image 608. On the other hand, if theuser gazes at the light source 606, local white balancing may instead beperformed using the color value of the light source 606 as the whitepoint, as shown in image 610. This may result in different colorsrendered in images 608, 610, even though the images may represent thesame physical scene. In some examples, the white point of the area nearthe user focus location may be adjusted to satisfy a creative need ofthe content creator.

For example, as represented in an image 608, the gaze analysis module410 may determine that the user's gaze is directed to a user focuslocation 612. The pixel modifier module 414 may adjust the white pointof image 608 based on an area around the user focus location 612.Because the area around the user focus location 612 has a colortemperature of approximately 2700 K, the white balance of image 608 maybe adjusted to a white point with a lower color temperature.

As another example, as represented in an image 610, the gaze analysismodule 410 may determine that the user's gaze is directed to a userfocus location 614. The pixel modifier module 414 may adjust the whitepoint of image 610 based on an area around the user focus location 614.Because the area around the user focus location 614 has a colortemperature of approximately 6500 K, the white balance of image 608 maybe adjusted to a white point with a higher color temperature.

In some examples, modifying the color mapping may include performingchromatic adaptation of the region of the image to counteract the effectof a phenomenon known as veiling glare. Veiling glare is an imperfectionof performance in optical instruments, such as lenses, that arises fromincoming light that strays from normal image-forming paths and reachesthe focal plane. As a result, noise may be superimposed on the imagesensed by a sensor or perceived by an eye, and the image may be degradedby loss of contrast and reduced definition. FIG. 7 illustrates a diagramof an example of veiling glare. As illustrated in FIG. 7 , an eye 702perceiving content on a display 704 through a lens 706 may perceive thecontent at a user focus location 708. High brightness content at anotherlocation 710 may result in the veiling glare phenomenon. An exampleimage 712 includes a portion 714 without the veiling glare effect and aportion 716 with the veiling glare effect. In some examples, the pixelmodifier module 414 may adjust the color values of at least part of theimage to counteract the veiling glare effect based on the user focuslocation. For example, the veiling glare effect may be generated by abright signal from a peripheral direction, as shown at the left side ofFIG. 7 . If it is known that the user is not looking directly at thebright signal, the color values of the part of the image correspondingto the bright signal may be adjusted to dim the bright signal. This mayreduce the veiling glare effect.

In some examples, due to optical design and hardware limitations, it maybe difficult to achieve uniform color across the field of view (FOV).Display hardware (e.g., organic light emitting diodes (OLEDs)) mayexhibit color changes when viewed from different viewing angles. Colorcorrection may be applied on a per frame basis across the display area,based on the user focus location (e.g., gaze direction). In someexamples, the color correction may be spatially slow, e.g., there may befew or no sharp corrections. In some examples, color correction may beperformed every 10° across the field of view (FOV) with a smoothtransition, such as a B spline surface, e.g., the surface 810 of FIG. 8.

In some examples, the pixel modifier module 414 may use user focuslocation (e.g., gaze direction) information to perform dynamic colorcorrection (DCC) and dynamic brightness correction (DBC) on an image.FIG. 8 illustrates a diagram showing examples of dynamic colorcorrection (DCC) and dynamic brightness correction (DBC). An image 802may represent an input image to which dynamic color correction (DCC) maybe applied. A color correction function may be represented by an image804. An image 806 may represent an input image to which dynamicbrightness correction (DBC) may be applied. A brightness correctionfunction may be represented by an image 808. A surface 810 may representexample correction functions.

In some examples, distortion (e.g., pupil swim) may be introducedinstability or exacerbated to achieve other benefits, such as higherresolution or more accurate color in the center field of view (FOV). Thepixel modifier module 414 may compensate for distortion instabilityassociated with changes in user focus location (e.g., gaze direction) bydynamic distortion correction (DDC). In some examples, dynamicdistortion correction (DDC) focuses on peripheral minor adjustments witha large spatial size. A large spatial size may correspond to a slowspatial frequency. If the correction is characterized by a Gaussianshape, the correction may have a wide shape, e.g., there may no highfrequency corrections. A base distortion may be present, e.g., topresent a barrel-shaped display pattern to achieve a regular angularspace through the lens. Dynamic color correction (DCC), dynamicbrightness correction (DBC), and/or dynamic distortion correction (DDC)may be applied on top of this base distortion correction.

Dynamic color correction (DCC), dynamic brightness correction (DBC),and/or dynamic distortion correction (DDC) may be performed by software,by a display driver integrated circuit (DDIC), and/or by a system on achip (SOC), which may include a central processing unit (CPU) and agraphical processing unit (GPU).

In some examples, color correction may be performed on a per-pixel basiswith a shared 3×3 transform in a graphics processor unit (GPU)compositor shader (e.g., linear gamma) on mobile devices. A personalcomputer (PC) platform may support more accurate corrections (e.g., 3Dcolor lookup). Such corrections may be too computationally expensive formobile graphics processor unit (GPU) implementations.

For dynamic color correction (DCC) and dynamic brightness correction(DBC), if the per-pixel 3×3 transform may be described as a combinationof two or three basis 3×3 transforms based on eccentricity, theper-pixel 3×3 transform may be handled with little, if any, costs incomputing resources. The interpolation function may be computationallycheap to evaluate. For example, linear or low order polynomials arecomputationally cheap to evaluate, while floating exponential orGaussian functions may be more computationally expensive. For dynamiccolor correction (DCC) and dynamic brightness correction (DBC), eachzone may have its own arbitrary 3×3 transform function. During runtime,it would be prohibitively computationally expensive to sample andinterpolate from hundreds of transforms. In some examples, a dedicatedpiece of silicon, e.g., a data processing unit (DPU) or a display driverintegrated circuit (DDIC) may facilitate sampling and interpolating fromthe 3×3 transforms.

In some cases, the one or more computing platforms 402 may becommunicatively coupled to the remote platform(s) 404. In some cases,the communicative coupling may include communicative coupling through anetworked environment 420. The networked environment 420 may include aradio access network, such as LTE or 5G, a local area network (LAN), awide area network (WAN) such as the Internet, and/or wireless LAN(WLAN), for example. It will be appreciated that these examples are notintended to be limiting, and that the scope of this disclosure includesexamples in which one or more computing platforms 402 and remoteplatform(s) 404 may be operatively linked via some other communicationcoupling. The one or more computing platforms 402 may be configured tocommunicate with the networked environment 420 via wireless or wiredconnections. In addition, in some examples, the one or more computingplatforms 402 are configured to communicate directly with each other viawireless or wired connections. Examples of one or more computingplatforms 402 may include, but are not limited to, smartphones, wearabledevices, tablets, laptop computers, desktop computers, Internet ofThings (IoT) devices, and/or other mobile or stationary devices. In someexamples, the system 400 may also include one or more hosts or servers,such as the one or more remote platforms 404 connected to the networkedenvironment 420 through wireless or wired connections. In some examples,remote platforms 404 may be implemented in or function as base stations,which may also be referred to as Node Bs or evolved Node Bs (eNBs). Insome examples, remote platforms 404 may include web servers, mailservers, application servers, etc. According to some examples, remoteplatforms 404 may be implemented as standalone servers, networkedservers, or an array of servers.

The one or more computing platforms 402 may include one or moreprocessors 426 for processing information and executing instructions oroperations. One or more processors 426 may be any type of general orspecific purpose processor. In some cases, multiple processors 426 maybe utilized. The one or more processors 426 may include one or more ofgeneral-purpose computers, special purpose computers, microprocessors,digital signal processors (DSPs), field-programmable gate arrays(FPGAs), application-specific integrated circuits (ASICs), andprocessors based on a multi-core processor architecture, as non-limitingexamples. In some cases, the one or more processors 426 may be remotefrom the one or more computing platforms 402, such as disposed within aremote platform like the one or more remote platforms 404.

The one or more processors 426 may perform functions associated with theoperation of the system 400, which may include, for example, precodingof antenna gain/phase parameters, encoding and decoding of individualbits forming a communication message, formatting of information, andoverall control of the one or more computing platforms 402, includingprocesses related to management of communication resources.

The one or more computing platforms 402 may further include or becoupled to the memory 418 (internal or external), which may be coupledto one or more processors 426, for storing information and instructionsthat may be executed by one or more processors 426. Memory 418 may beone or more memories and of any type suitable to the local applicationenvironment, and may be implemented using any suitable volatile ornonvolatile data storage technology such as a semiconductor-based memorydevice, a magnetic memory device and system, an optical memory deviceand system, fixed memory, and removable memory. For example, memory 418may include any combination of random access memory (RAM), read onlymemory (ROM), static storage such as a magnetic or optical disk, harddisk drive (HDD), or any other type of non-transitory machine orcomputer readable media. The instructions stored in memory 418 mayinclude program instructions or computer program code that, whenexecuted by one or more processors 426, enable the one or more computingplatforms 402 to perform tasks as described herein.

In some examples, the one or more processors 426 may include one or moregraphics processing units (GPUs). GPUs may include more arithmetic logicunits (ALUs) compared with CPUs. GPUs may be suitable for performingbillions of repetitive arithmetic operations. For example, GPUs may besuitable to address problems that may be expressed as data-parallelcomputations, e.g., problems in which the same program is executed onmany data elements in parallel, with a high ratio of arithmeticoperations to memory operations.

In some examples, a gaze-aware graphics processing unit (GPU)architecture may determine what operations arithmetic logic units (ALUs)are running, conserving bandwidth (e.g., data needed to transfer frommemory to the graphics processing unit (GPU)). Operations close to theeye gaze may be more important on a frame-by-frame basis. A graphicsprocessing unit (GPU) may be able to determine a level to whichrendering occurs, potentially obviating the need for pixel-leveloperations if they are farther away from the eye gaze.

In some examples, one or more computing platforms 402 may also includeor be coupled to one or more antennas 430 for transmitting and receivingsignals and/or data to and from one or more computing platforms 402. Theone or more antennas 430 may be configured to communicate via, forexample, a plurality of radio interfaces that may be coupled to the oneor more antennas 430. The radio interfaces may correspond to a pluralityof radio access technologies including one or more of LTE, 5G, WLAN,Bluetooth, near field communication (NFC), radio frequency identifier(RFID), ultrawideband (UWB), and the like. The radio interface mayinclude components, such as filters, converters (for example,digital-to-analog converters and the like), mappers, a Fast FourierTransform (FFT) module, and the like, to generate symbols for atransmission via one or more downlinks and to receive symbols (forexample, via an uplink).

FIG. 9 is a flow diagram illustrating an example method 900 for usinggaze-aware tone mapping and chromatic adaptation, according to variousexamples. In various examples, the method 900 is performed by a device(e.g., the color correction system 400 of FIG. 4 ). In some examples,the method 900 is performed by processing logic, including hardware,firmware, software, or a combination thereof. The method 900 may beperformed by a processor executing code stored in a non-transitorycomputer-readable medium (e.g., a memory). Briefly, in various examples,the method 900 may include determining a user focus location within aphysical environment and receiving an image corresponding to thephysical environment. A visual property of the physical environmentcorresponding to the user focus location may be determined. A colormapping of a region of the image may be modified based on the determinedvisual property to generate an output image.

As represented by block 910, in various examples, the method 900 mayinclude determining a user focus location within a physical environment.For example, the color correction system 400 may determine a location inthe physical environment to which the user's attention is directed. Gazedirection may be used as a proxy for the user's attention. For example,the color correction system 400 may determine that the user's attentionis focused on a location at which the user's gaze is directed. Asrepresented by block 910 a, in some examples, the gaze analysis module410 may determine the user focus location based on a gaze direction. Thegaze analysis module 410 may determine the gaze direction of the userbased on image data representing the user's eyes. For example, the gazeanalysis module 410 may receive image data from user-facing cameras orimage sensors. The image data may represent an image or images thatinclude pixels representing the user's pupils. Based on the position ofthe user's pupils, the gaze analysis module 410 may determine thedirection of the user's gaze. In some examples, the user's eyes may beilluminated, for example, by LASER or near infrared (nIR) illuminationsources, such as light emitting diodes (LEDs). The image represented bythe image data may include pixels representing reflections from theuser's eyes (e.g., glints) that may be used to determine the directionof the user's gaze.

As represented by block 920, in various examples, the method 900 mayinclude receiving an image corresponding to the physical environment.For example, For example, the color analysis module 412 may receiveimage data representing the physical environment from a world-facingcamera or image sensor. Due to optical design and hardware limitations,the user's view of the physical environment may differ from the image.For example, it may be difficult to achieve uniform color across thefield of view of a displayed representation of the physical environment.For example, display hardware, such as organic light emitting diodes(OLEDs), may exhibit changes in perceived color when viewed fromdifferent viewing angles. Optical layers and/or lenses between thedisplay and the human eye may also contribute to color distortion. Asanother example, viewing optics in virtual reality (VR) systems (e.g.,pancake optics) may exhibit ghosting and veiling glare issues due tocomplex optical layers and coatings.

As represented by block 930, in various examples, the method 900 mayinclude determining a visual property of the physical environmentcorresponding to the user focus location. In some examples, asrepresented by block 930 a, the visual property may include a colorvalue characterizing the user focus location. For example, the coloranalysis module 412 may analyze color values associated with one or morepixels in the image data to determine a color value associated with thelocation in the physical environment to which the user's gaze isdirected. In some examples, the visual property may include a brightnessvalue characterizing the user focus location. For example, brightnessmay be determined based on RGB values of pixels in a selected area.

In some examples, as represented by block 930 b, the method 900 mayinclude determining a size of the user focus location. For example, thecolor analysis module 412 may select a size of the area that is to serveas a basis for tone mapping and/or chromatic adaptation. In someexamples, the color analysis module 412 may apply an averaging orfiltering function to the image data representing the physicalenvironment. For example, the color analysis module 412 may apply afiltering function to account for noise and/or natural micro saccadefrom the eye tracker. In some examples, the color analysis module 412may differentiate between different eye movements such as saccade, microsaccade, slow pursuit, and/or vestibulo ocular reflex (VOR). The coloranalysis module 412 may apply different filtering functions to accountfor each type of eye movement. For example, the color analysis module412 may apply little or no averaging to the image data in the case ofsaccade so that changes may be applied more quickly.

As represented by block 940, in various examples, the method 900 mayinclude modifying a color mapping of a region of the image correspondingto the physical environment based on the determined visual property togenerate an output image. For example, the pixel modifier module 414 mayuse the color values from the area around the user focus location toperform color correction on a region of the image.

In some examples, as represented by block 940 a, modifying the colormapping may include adjusting a tone mapping of the region of the image.For example, the pixel modifier module 414 may perform dynamicbrightness adjustment, as described herein in connection with FIG. 5 .

Due to optical design and hardware limitations, it may be difficult toachieve uniform color across the field of view (FOV). Display hardware(e.g., organic light emitting diodes (OLEDs)) may exhibit color changeswhen viewed from different viewing angles. In some examples, brightnesscorrection may be implemented as part of color correction. Brightnesscorrection may be achieved by emission duty or persistence adjustmentand/or backlight spatial adjustment. Brightness correction may bespatially slow, e.g., there may be few or no sharp corrections. In someexamples, brightness correction may be performed every across the fieldof view (FOV) with a smooth transition, such as a B spline surface.

In some examples, as represented in block 940 b, modifying the colormapping may include adjusting a chromatic mapping of the region of theimage. For example, the pixel modifier module 414 may perform localwhite balancing (e.g., adjust a white point) based on an area around theuser focus location, as described herein in connection with FIG. 6 .

In some examples, due to optical design and hardware limitations, it maybe difficult to achieve uniform color across the field of view (FOV).Display hardware (e.g., organic light emitting diodes (OLEDs)) mayexhibit color changes when viewed from different viewing angles. Colorcorrection may be applied on a per frame basis across the display area,based on the user focus location (e.g., gaze direction). In someexamples, the color correction may be spatially slow, e.g., there may befew or no sharp corrections. In some examples, color correction may beperformed every 10° across the field of view (FOV) with a smoothtransition, such as a B spline surface.

In some examples, the pixel modifier module 414 may use user focuslocation (e.g., gaze direction) information to perform dynamic colorcorrection (DCC) and dynamic brightness correction (DBC) on an image, asdescribed herein in connection with FIG. 8 .

In some examples, as represented in block 940 c, the method 900 mayinclude modifying a set of pixels of the image based on the user focuslocation to correct distortion. Distortion (e.g., pupil swim) may beintroduced instability or exacerbated to achieve other benefits, such ashigher resolution or more accurate color in the center field of view(FOV). The pixel modifier module 414 may compensate for distortioninstability associated with changes in user focus location (e.g., gazedirection) by dynamic distortion correction (DDC). In some examples,dynamic distortion correction (DDC) focuses on peripheral minoradjustments with a large spatial size. A base distortion may be present,e.g., to present a barrel-shaped display pattern to achieve a regularangular space through the lens. Dynamic color correction (DCC), dynamicbrightness correction (DBC), and/or dynamic distortion correction (DDC)may be applied on top of this base distortion correction.

Dynamic color correction (DCC), dynamic brightness correction (DBC),and/or dynamic distortion correction (DDC) may be performed by software,by a display driver integrated circuit (DDIC), and/or by a system on achip (SOC), which may include a central processing unit (CPU) and agraphical processing unit (GPU).

In some examples, color correction may be performed on a per-pixel basiswith a shared 3×3 transform in a graphics processor unit (GPU)compositor shader (e.g., linear gamma) on mobile devices. A personalcomputer (PC) platform may support more accurate corrections (e.g., 3Dcolor lookup). Such corrections may be too computationally expensive formobile graphics processor unit (GPU) implementations.

For dynamic color correction (DCC) and dynamic brightness correction(DBC), if the per-pixel 3×3 transform may be described as a combinationof two or three basis 3×3 transforms based on eccentricity, theper-pixel 3×3 transform may be handled with little, if any, costs incomputing resources. The interpolation function may be computationallycheap to evaluate. For example, linear or low order polynomials arecomputationally cheap to evaluate, while floating exponential orGaussian functions may be more computationally expensive. For dynamiccolor correction (DCC) and dynamic brightness correction (DBC), eachzone may have its own arbitrary 3×3 transform function. During runtime,it would be prohibitively computationally expensive to sample andinterpolate from hundreds of transforms. In some examples, a dedicatedpiece of silicon, e.g., a data processing unit (DPU) or a display driverintegrated circuit (DDIC) may facilitate sampling and interpolating fromthe 3×3 transforms.

In the foregoing description, various examples are described, includingdevices, systems, methods, and the like. For the purposes ofexplanation, specific details are set forth in order to provide athorough understanding of examples of the disclosure. However, it willbe apparent that various examples may be practiced without thesespecific details. For example, devices, systems, structures, assemblies,methods, and other components may be shown as components in blockdiagram form in order not to obscure the examples in unnecessary detail.In other instances, well-known devices, processes, systems, structures,and techniques may be shown without necessary detail in order to avoidobscuring the examples.

The figures and description are not intended to be restrictive. Theterms and expressions that have been employed in this disclosure areused as terms of description and not of limitation, and there is nointention in the use of such terms and expressions of excluding anyequivalents of the features shown and described or portions thereof. Theword “example” is used herein to mean “serving as an example, instance,or illustration.” Any embodiment or design described herein as “example”is not necessarily to be construed as preferred or advantageous overother embodiments or designs.

Although the methods and systems as described herein may be directedmainly to digital content, such as videos or interactive media, itshould be appreciated that the methods and systems as described hereinmay be used for other types of content or scenarios as well. Otherapplications or uses of the methods and systems as described herein mayalso include social networking, marketing, content-based recommendationengines, and/or other types of knowledge or data-driven systems.

1. A system, comprising: a processor; and a memory storingprocessor-executable instructions that, when executed by the processor,cause the processor to: determine a user focus location within aphysical environment; receive an image corresponding to the physicalenvironment; determine a visual property of the physical environmentcorresponding to the user focus location; and modify a color mapping ofa region of the image based on the determined visual property togenerate an output image.
 2. The system of claim 1, wherein the memorystores further processor-executable instructions for determining theuser focus location based on a gaze direction.
 3. The system of claim 1,wherein the visual property comprises a color value characterizing theuser focus location.
 4. The system of claim 1, wherein the memory storesfurther processor-executable instructions for determining a size of theuser focus location.
 5. The system of claim 1, wherein the memory storesfurther processor-executable instructions for modifying a color mappingof a region of the image based on the determined visual property togenerate an output image at least in part by adjusting a tone mapping ofthe region of the image.
 6. The system of claim 1, wherein the memorystores further processor-executable instructions for modifying a colormapping of a region of the image based on the determined visual propertyto generate an output image at least in part by adjusting a chromaticmapping of the region of the image.
 7. The system of claim 1, whereinthe memory stores further processor-executable instructions formodifying a set of pixels of the image based on the user focus locationto correct distortion.
 8. A method, comprising: determining a user focuslocation within a physical environment; receiving an image correspondingto the physical environment; determining a visual property of thephysical environment corresponding to the user focus location; andmodifying a color mapping of a region of the image based on thedetermined visual property to generate an output image.
 9. The method ofclaim 8, further comprising determining the user focus location based ona gaze direction.
 10. The method of claim 8, wherein the visual propertycomprises a color value characterizing the user focus location.
 11. Themethod of claim 8, further comprising determining a size of the userfocus location.
 12. The method of claim 8, wherein modifying a colormapping of a region of the image based on the determined visual propertyto generate an output image comprises adjusting a tone mapping of theregion of the image.
 13. The method of claim 8, wherein modifying acolor mapping of a region of the image based on the determined visualproperty to generate an output image comprises adjusting a chromaticmapping of the region of the image.
 14. The method of claim 8, furthercomprising modifying a set of pixels of the image based on the userfocus location to correct distortion.
 15. A nontransitory computerreadable storage medium comprising an executable that, when executed,instructs a processor to: determine a user focus location within aphysical environment; receive an image corresponding to the physicalenvironment; determine a visual property of the physical environmentcorresponding to the user focus location; and modify a color mapping ofa region of the image based on the determined visual property togenerate an output image.
 16. The computer readable storage medium ofclaim 15, wherein the executable further causes the processor todetermine the user focus location based on a gaze direction.
 17. Thecomputer readable storage medium of claim 15, wherein the visualproperty comprises a color value characterizing the user focus location.18. The computer readable storage medium of claim 15, wherein theexecutable further causes the processor to determine a size of the userfocus location.
 19. The computer readable storage medium of claim 15,wherein the executable further causes the processor to modify a colormapping of a region of the image based on the determined visual propertyto generate an output image at least in part by adjusting at least oneof a tone mapping or a chromatic mapping of the region of the image. 20.The computer readable storage medium of claim 15, wherein the executablefurther causes the processor to modify a set of pixels of the imagebased on the user focus location to correct distortion.